Application of an Object-Oriented Parallel Run-Time System to a Grand Challenge 3D Multi-Grid Code

We have taken a Grand Challenge 3d Multi-Grid code, initially developed on the Cray C-90 and subsequently parallelized for MPPs, and implemented it using the DUDE object-oriented, run-time system which combines both task and data parallelism. The Grand Challenge 3d Multi-Grid code, QGMG (Quasi-Geostrophic Multi-Grid), solves the quasigeostrophic equations which describe the nonlinear dynamics of rotating, stably stratified fluids; this is the relevant regime for most planetary-scale motions in the Earth’s atmosphere and ocean [17]. This code was originally developed on the Cray C-90 and runs at 6 Gflops on all 16 processors. Over the last year the code was implemented it in a portable parallel way on most of todays MPPs (Massively Parallel Processors) [5]. This was done via domain decomposition and message passing using PVM and MPI. Currently, on 256 processors of the Cray T3D the code runs at almost 4 Gflops and on 256 processors of the IBM SP2 at 5 Gflops. We have taken this parallel code and implemented it using the DUDE object-oriented, run-time system [9]. The QGMG solver is a challenging application for two reasons. First, as in all multigrid solvers, the most straight forward implementation requires that most of the processors idle at barrier synchronizations. Secondly, the QGMG solver is an example of an application that requires both task and data parallelism. In this application, two multigrids (task parallelism) must be solved and each multigrid solver contains data-parallelism. To address the first challenge, DUDE loosens the requirement that all processes must wait at barriers. In fact we have been able to eliminate barriers in critical loops. Much like in dataflow computations, processes may go on to the next operations when the data dependencies are satisfied. This is done by providing a dependence vector to the run-time system. To address the second problem, we have built a run-time system that provides integrated task parallelism and data parallelism. This combined approach is achieved by using object parallelism.